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The Safety Performance of Airborne Separation (SPAS) study is a suite of Monte Carlo 
simulation experiments designed to analyze and quantify safety behavior of airborne 
separation. This paper presents results of preliminary baseline testing. The preliminary 
baseline scenario is designed to be very challenging, consisting of randomized routes in 
generic high-density airspace in which all aircraft are constrained to the same flight level. 
Sustained traffic density is varied from approximately 3 to 15 aircraft per 10,000 square 
miles, approximating up to about 5 times today’s traffic density in a typical sector. Research 
at high traffic densities and at multiple flight levels are planned within the next two years. 
Basic safety metrics for aircraft separation are collected and analyzed. During the 
progression of experiments, various errors, uncertainties, delays, and other variables 
potentially impacting system safety will be incrementally introduced to analyze the effect on 
safety of the individual factors as well as their interaction and collective effect. In this paper 
we report the results of the first experiment that addresses the preliminary baseline 
condition tested over a range of traffic densities. Early results at live times the typical traffic 
density in today’s NAS indicate that, under the assumptions of this study, airborne 
separation can be safely performed. In addition, we report on initial observations from an 
exploration of four additional factors tested at a single traffic density: broadcast surveillance 
signal interference, extent of intent sharing, pilot delay, and wind prediction error. 
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I. Introduction 

T he Joint Planning and Development Office (JPDO), lead organization for the development and evaluation of 
the Next Generation Air Transportation System (NextGen) concept, has directed that understanding the impact 
of new operational procedures and capabilities on the safe operation of the National Airspace System (NAS) is of 
foremost importance. The NextGen Concept of Operations 1 incorporates various types of Airborne Separation 
Assistance System (ASAS) applications. ASAS applications, as defined by the FAA/Eurocontrol 2 Action Plan 1, 
include the categories of situational awareness, spacing, delegated separation, and full responsibility for self- 
separation. Before procedures are written and the roles of humans and automation are identified, it is necessary to 
evaluate the safety performance of these applications. 

Airborne separation is enabled by advanced decision-aiding automation integrated into aircraft avionics. 
The automation is designed to detect conflicts between aircraft, help pilots resolve them, and provide conflict- 
preventative support for conflict-free maneuvering. These decision aides rely on broadcast surveillance information 
that includes aircraft velocity vectors and limited flight plan information. To assess safety, it is imperative to 
understand the effects that changes in traffic levels, information accuracy, prediction errors, datalink range and 
content, and other uncertainties and variables may have on the distributed automated systems. In particular, the 
implications of such conditions to the safe operation of NextGen need to be evaluated with comprehensive models 
that consider all separation assurance applications. 

The Safety Performance of Airborne Separation (SPAS) simulation study is designed to investigate the effect of 
traffic demand on safety performance of distributed airborne separation in the presence of various errors, 
uncertainties, delays, and other variables potentially impacting system safety. The study is comprised of a series of 
simulation experiments that progressively characterize and quantify safety in en route airspace using established 
criteria and safety metrics. The intention is to develop a broad set of generalizable safety performance data for use in 
safety analyses and for eventual comparison with similar studies focused on ground-based separation. 

The primary goal of this preliminary baseline phase of the SPAS study is to develop an initial understanding of 
the safety performance of airborne separation in a baseline condition as traffic level is significantly increased. An 
additional goal is to collect initial exploratory data on the impact of several safety-impacting variables to help guide 
more detailed experimentation planned for later phases. The experiment is conducted in a distributed batch 
simulation in the Air Traffic Operations Lab (ATOL) of the NASA Langley Research Center. The Autonomous 
Operations Planner (AOP), an established research software implementation of ASAS automation, is used to provide 
the airborne separation function. Testing is performed under the rigorous conditions posed by randomly generated, 
highly constrained traffic scenarios. The test environment selected for this initial phase is a generic circular scenario 
with random trajectories and all flight paths (including conflict resolution paths) restricted to a single altitude. The 
random traffic scenarios are generated such that they exhibit a high rate of conflicts whose lateral encounter angles 
vary over nearly the full range of values. 

Subsequent phases of the SPAS study will be expanded to include three-dimensional (3D) airspace, and traffic 
structures will be constrained to resemble real traffic patterns found in the NAS. These multi-flight-level scenarios 
will exhibit aircraft conflict geometries more like those found in real traffic conditions, but they will be modified to 
increase the traffic density and conflict frequency. Through the incremental incorporation of various errors, 
uncertainties, delays, and other variables, and through the testing of ASAS capabilities designed to compensate for 
these factors, the complete SPAS study will provide some of the key data needed for a comprehensive safety 
analysis of airborne separation concepts and for comparison to other methods of separation management. 

II. Background 

Safety evaluation of new Air Traffic Management concepts and technologies involves modeling the often very 
complex relationships between individual subsystems in order to assess system reliability and robustness. The effect 
of subsystem failures or degradation on the system performance needs to be assessed by analytical tools such as 
fault tree analyses and other risk assessment techniques that require quantitative safety data as input 3 ’ 4 . The overall 
goal and scope of the SPAS study is to begin to understand and quantify some safety performance aspects of ASAS 
applications using system uncertainties and prediction error models that will eventually contribute to the safety 
analysis of NextGen. 

Until now, safety evaluation of ASAS applications have for the most part been based on low fidelity studies, 
seldom including assessment of system uncertainties. One such study 5 addressed the “feasibility” of two conflict 
detection and resolution methods in a quantitative simulation study that measured safety, efficiency and stability of 
the system. Safety was evaluated in terms of the ability of the two methods to successfully resolve all conflicts in the 
airspace. The study emphasized that both ASAS methods resolved all conflicts with only small path deviations and 
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that safety was not compromised. An additional stability metric reflected the potential propagation of conflicts as a 
result of decentralized resolution methods. The study posed that lack of stability would represent an efficiency 
degradation if the number of path deviations is increased as a result of additional conflict resolutions. The opposite 
was observed in the study, which concluded that both methods provide very efficient resolutions. A later study 6 , 
intending to compare the performance of centralized and decentralized air traffic separation methods, revisited the 
notion of stability and efficiency metrics in a low fidelity model of three separation techniques. 

The safety of ground-based automated separation was the subject of a recent study 7 that presented a preliminary 
fault-tree analysis of the effect of some system faults on the safety of a centralized separation system. The analysis 
did not consider system uncertainties or performance degradation, nor did it consider prediction errors. Based on 
assumed fault rates, the study suggested that system safety would be preserved. In particular, it concluded that under 
the assumptions made, the system may be able to achieve safety levels higher than today’s levels. 

III. Simulation Platform and Scenarios Description 
A. The Airspace & Traffic Operations Simulation 

The SPAS experiments are being conducted at the Air Traffic Operations Laboratory (ATOL) at NASA Langley 
Research Center utilizing a distributed simulation platform that includes a grid of aircraft simulators interconnected 
through a High 
Level Architecture 
communication 
infrastructure. The 
simulation 
platform, known as 
the Airspace & 

Traffic Operations 
Simulation 
(ATOS) 8 depicted 
in Fig. 1, can be 
used for batch 
Monte Carlo 

studies as well as 
real time human-in 

the-loop experiments. The ATOS is comprised of a number of real time, medium fidelity aircraft simulators referred 
to as Aircraft Simulation for Traffic Operations Research (ASTOR), and a Simulation Manager. Each ASTOR 
represents a single aircraft and is modeled by a 6-degree of freedom dynamic model, a Flight Management System 
(FMS), Mode S Automatic Dependent Surveillance - Broadcast (ADS-B) datalink capability, and a prototype 
airborne separation automation that is described in the next section. For batch simulations, all ASTORS are “flown” 
by a computer-based pilot model (PM) that performs the basic pilot actions for airborne conflict management. The 
PM was designed to automatically take over the controls of an ASTOR with no interference to the system and 
without the aid of a human pilot. The PM design is composed of three parts: a sensory input model, a rule-based 
decision model, and an actuator response model. The sensory input model gathers information and passes it to the 
rules model. The rules model determines the PM actions and commands the pilot action model to execute that 
action. Both the sensory input and actuator response models are governed by a simple time delay algorithm. A 
normally distributed time delay is used to configure the PM “personality” as one of several profiles which dictates 
its responsiveness level. The preliminary baseline experiment uses a “normal” PM with a mean and maximum 
sensory delay time 500 and 5000 milliseconds respectively. 

Other ATOS modules include a background traffic generator capability called TMX 9 that models lower fidelity 
air traffic with full airborne separation capabilities and provides a plan view display for traffic visualization. The 
communication infrastructure allows the ATOS capabilities to be extended by connecting with other real time or 
batch simulation environments. 
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Figure 1. SPAS Simulation Platform 
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B. Airborne Conflict Management System 


The intent of this study is to document whether adequate safety performance of airborne separation is achievable 
without regard to the optimality of the specific ASAS approach and algorithms used. A brief description of the 
system chosen for the study follows. 

The airborne conflict management system used in this experiment series is the Autonomous Operations Planner 
(AOP), a NASA-developed research model of an airborne automation system built for the study of advanced 
distributed air-ground operational concepts 10 . The AOP provides an integrated suite of capabilities for managing 
traffic conflicts and trajectory changes from the flight deck perspective, including conflict detection, resolution, 
prevention, and trajectory constraint conformance. These capabilities are developed to a level sufficient for research 
of complete airborne responsibility for self-separation as defined by the Principles of Operation of ASAS, Category 
4 2 . In the JPDO NextGen Concept of Operations, they correspond to the functions of Separation Management and 
aspects of Trajectory Management 1 . 

The intent-based conflict detection (CD) function of AOP uses state and intent data received from other traffic 
aircraft over ADS-B in combination with ownship state data, autoflight mode settings, and flight plan information to 
deterministically predict future losses of separation. For this study, the autoflight mode remained coupled to the 
FMS for lateral navigation. The CD look-ahead horizon was set to 10 minutes and the separation criterion to 
5.2 nautical miles. A capability exists in AOP to apply CD buffers tailored to individual flight segments to account 
for trajectory prediction error sources such as wind-field forecasts 11 . Although the baseline runs of this preliminary 
study excluded most error sources and corresponding buffers, they are planned for inclusion in upcoming stages of 
the experiment series. AOP also has a second, independent, CD system that uses state -vector projections to detect 
flight crew blunders and prediction faults of the intent-based CD system. A complete system is envisioned to 
contain both state-based and intent-based CD systems for redundancy and safety 12 . Conflict alerting is modeled 
after the multi-alert-level approach recommended by RTCA 13 . The timing of alerts between aircraft are staggered as 
a method for incorporating a right-of-way rule set (i.e. priority rules), based roughly on the set used in Visual Flight 
Rules 14 . In the current study, the simple computer-based pilot model reacted to “low level alerts” within a normally 
distributed timing scheme defined by response type. When alerted to a conflict, the pilot model’s action was to 
request a fully-conflict-free resolution trajectory from AOP. 

For conflict resolution (CR), AOP contains both strategic and tactical capabilities. Tactical CR refers to open- 
loop vectors or altitude changes to solve conflicts with no predetermined reconnection plan. Along with the state- 
based CD, the tactical CR capability was disabled to allow the focus of the current study to be on distributed 
strategic systems. Strategic CR refers to the single action of modifying the flight plan such that the conflict is solved 
and the aircraft reconnects to the previous trajectory. In the current study, strategic CR solutions were purposefully 
constrained to lateral maneuvers only. In generating solutions, AOP’s strategic CR system takes into account all 
known trajectory constraints, including trajectories of nearby traffic, airspace hazards, ownship performance limits, 
and required time of arrival (RTA) constraints. A genetic algorithm is employed to search within a set of pre- 
defined geometric patterns to generate CR trajectories that simultaneously accommodate these constraints 15 . Once a 
suitable CR trajectory is calculated, and it is verified conflict free for 20 minutes based on current traffic data (this 
time is not fully applicable to the current traffic scenario because new aircraft are continuously being created during 
the test runs), it is presented to the flight crew for review. In the current study, the pilot model was programmed to 
execute the first available CR trajectory. Immediately upon execution of the route, ADS-B broadcasts the new 
ownship intent as a series of trajectory change points. 

AOP also includes functions for conflict prevention, including at-a-glance maneuver restriction symbology for 
the flight crew and support for tactical / strategic trajectory probing (also known as provisional CD). These 
functions were not required for this study and were therefore disabled. 
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IV. Experiment Design 

A real time scenario generation tool was developed for this experiment that creates initially straight, random 
aircraft routes traversing a circular test area representing a generic, high density, en route sector. In this 
environment, ASTOR aircraft are introduced into the scenario on the random routes, traverse the airspace while 
detecting and solving conflicts, and exit the airspace while proceeding in the direction of the initial downstream 
waypoint constrained by a RTA. A conflict is defined as a predicted Loss of Separation (LOS) between two aircraft. 
The applicable separation standard used is 5 NM. After leaving the test area and before reaching the RTA waypoint, 
metrics for the flight are recorded and the ASTOR is reinitialized with a new random route and reintroduced to the 
test area. The number of 
independent ASTOR aircraft (i.e. 
each a separate networked 
computer) operating 

simultaneously in the simulation 
ranged from 14 to 72. 

The test area, depicted in Figure 
2, is modeled as two concentric 
circles of 80 NM and 160 NM 
radii, respectively. The region 
between the circles is referred to as 
the initialization region, and the 
region within the inner circle is the 
test region. All aircraft are created 
just outside the initialization region 
and their paths traverse the test 
region where experiment metrics 
are collected. Because AOP 
functions were purposefully 
disabled in the initialization ring, 
conflicts fully outside the test region were excluded from analysis, as were LOS events where either aircraft was 
outside the test region. The relative dimensions of the two circles were chosen to allow approximately 10 minutes of 
look-ahead time for the strategic conflict detection and 20 minutes of flying time in the test region. The target traffic 
demand was maintained within the limitations of the simulation platform, typically with a standard deviation of ± 3 
aircraft or less. 

The initial route was defined by three fixes. The first fix was generated as a [latitude, longitude] pair 
randomly placed on the perimeter of the outer circle. The second fix, located randomly on the opposite side of the 
inner circle and also defined as a [latitude, longitude] pair, was chosen to ensure that all initial trajectories intersect 
the test region. For all newly initiated aircraft entering the initialization region, the scenario generation logic ensured 
that their respective initial positions were at least 10 NM apart and that their respective paths would not intersect in 
this region. As stated earlier, any conflicts occurring fully outside the test region were ignored and not solved. The 
third fix was created outside the test region, approximately 500 NM from the first fix along the straight line route 
created by the first and second fixes. An RTA constraint was placed on the third fix to represent flow-managed 
operations. Conflict resolutions were required to maintain RTA conformance when possible. All trajectories were 
placed at the same altitude and only lateral resolutions were exercised in order to maintain the desired traffic 
densities. As mentioned above, all aircraft were terminated as soon as they exited the test region and reinitialized for 
reinsertion on a new random route. 

Table 1. Traffic Demand for Busiest Altitude in Sectors ZOA31 and ZOB46 on 19 Feb 2004 



Median Sector 
(ZOA31) 16,624 sq NM 
( Oakland ) 

Dense Sector 
(ZOB46) 5,959 sq NM 
( Cleveland ) 


Mean 

Peak 

Mean 

Peak 

Sector density at FL310 

3 

5 

5 

10 

Normalized 
IX Density per 10000 NM 2 

1.8 

3.0 

8.45 

16.85 
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Traffic demand levels in the NAS were analyzed using ETMS flight data from 19 February 2004, the good 
weather, high-traffic day selected by the JPDO and the FAA to represent baseline “IX” density. An analysis using 
the NASA ACES 16 analysis tool was conducted to determine the traffic count for every high altitude sector in the 
United States, at each flight level, for the 24 hour period. From this data, the mean and maximum traffic count for 
FL310 (the busiest altitude within the selected sectors) was determined for the median sector ZOA31 and a very 
busy en route sector ZOB46. Table 1 shows the “IX” traffic demand from 19 February 2004 in these two sectors: 
ZOA31, typical traffic density and similar in size to the test scenario, and ZOB46, a much smaller sector with very 
dense and complex traffic conditions. The traffic levels were then normalized to produce densities based on 
10,000 NM 2 . 


V. Experiment Results 

A. Input Data: Analysis of the Conditions Produced in the Test Scenarios 

The experiment consisted of six sets of simulation runs as indicated in Table 2. Each run was six hours of 
continuous simulation at the sustained traffic density level. Each run set consisted of six independent replicates of 
the six-hour run, except for the two highest density conditions, which had only two replicates each. 


Table 2: Summary of Simulation Runs 


Run 

Set 

Sustained 

Mean 

Density per 
10000 NM 2 

Standard 

Deviation 

Peak 

Density per 
10000 NM 2 

Total 

Number of 
Simulation 
Hours 

Total 
Number 
of Flights 

Total 
Number 
of Flight 
Hours 

Total 

Number 

of 

Conflicts 

1 

3.45 

0.59 

4.97 

36 

881 

237.27 

195 

2 

6.11 

0.83 

8.29 

36 

1527 

418.6 

550 

3 

8.61 

0.97 

11.44 

36 

2195 

544.57 

1018 

4 

11.64 

1.23 

15.34 

36 

3000 

797.17 

1788 

5 

15.24 

1.49 

19.31 

12 

1302 

347.45 

963 

6 

17.18 

1.54 

21.39 

12 

1560 

399.08 

1256 

Totals 

10465 

2744.14 

5770 


The peak and sustained mean density for each replicate was computed and normalized to 10000 NM‘. As can be 
seen, the sustained traffic densities for the different sets either match or surpass the values indicated in Table 1. The 
table also includes the total number of simulation hours, unique generated flights, cumulative flight hours and total 
number of conflicts for each density set. 

The mean number of traffic conflicts 
generated within the test region per 
flight hour as a function of normalized 
traffic density is shown in Fig. 3. As the 
graph shows, the frequency of conflicts 
per flight hour increases with traffic 
density ranging from an average of 0.82 
conflicts per flight hour for a density of 
3.45 aircraft per 10000 NM 2 to 3.15 
conflicts per flight hour for densities of 
17.8 aircraft per 10000 NM 2 . The high 
frequency of conflicts is a result of the 
highly constrained, randomly generated 
traffic scenarios. 

Correlation of this metric with the 
number of potential conflicts predicted 
in real air traffic conditions is not easy since the data is not readily available. The frequency of predicted conflicts 
could be loosely equated to the number of times route or speed adjustments are issued to aircraft to avoid potential 
separation violations. Controller instructions are communicated via voice, making a comparison metric difficult to 
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retrieve. A study conducted to develop air traffic scenarios resembling real traffic conflict conditions using Host 
Computer System tracks of the Memphis Air Route Traffic Control Center (ARTCC) estimated a total of 203 
predicted conflicts for 4332 flight hours 17 . That would represent about 0.05 conflicts per flight hour for an estimated 
mean traffic density of 2 to 4 aircraft per 10000 NM 2 (at the densest flight level in the ARTCC). 



Figure 4 


Conflict Encounter Angle 


Another important property used 
to characterize the conflict traffic 
scenarios is the encounter angle 
distribution. The conflict encounter 
angles observed in the random traffic 
scenarios used in this experiment are 
depicted in Figure 3.. The graph shows 
that the proportion of conflict angles is 
maintained within each group for all 
tested densities. The percentage of 
small angle (0 to 30 degrees) conflicts 
is less than all other angle brackets. 

However, the actual count of low 
angle conflicts observed is 163 per 
1000 flight hours as compared to 14 
small angle conflicts per 1000 flight 
hours observed in the extracted 

scenarios reported in the study above mentioned 17 . The Memphis ARTCC study, which did not involve random 
routing, indicates that almost 30% (approximately 60 conflicts) of all conflicts angles are between 0 and 30 degrees, 
which are considered the most challenging class of conflicts, and less than 10% are between 120 and 150 degrees. 
All other angle brackets are evenly 
represented in the data. In the current 
experiment, low angle conflict 
frequency was reduced as a result of 
measures taken to minimize conflicts in 
the initialization ring. 

The predicted separation at the 
closest point of approach (CPA) gives a 
measure of the magnitude of the 
predicted intrusions and the complexity 
of the traffic scenarios. Figure shows 
the distribution of horizontal distances 
at the predicted CPA for each traffic 
density. The distances are roughly 
evenly distributed between 0 and 5 NM 
for low traffic densities. As traffic 
density increases (and conflict count 
increases), predicted CPAs are more 
prevalent between 3 and 5 NM, which 
is reasonable considering the greater 
area bounded by these radii and 

therefore the greater probability of containing a predicted CPA. A small number of conflicts were recorded between 
5 and 5.1 NM as a result of a conflict detection buffer. 



Figure 5 


Predicted Distance at CPA 


B. Output Data: Analysis of the Experiment Outcomes 

The conflicts shown in Fig. 5 were detected and resolved using the technique of distributed airborne separation. 
Figure 6 presents the distribution of actual CPAs (i.e. as flown after resolution maneuvers were implemented) for 
these same aircraft pairs. Non-conflicting aircraft pairs were not included in this data. The data show that all 
conflicts were solved, with the exception of three borderline cases. The CPAs of these cases were 4.986, 4.989, and 
4.999 NM (i.e. maximum protected zone penetration of 85 feet). In two of these cases, an aircraft was executing a 
turn at the time of LOS. These two borderline conflicts were not detected as a result of modeling the turns as a 

7 

American Institute of Aeronautics and Astronautics 


series of straight segments, whereas the ASTOR flies curved turns. In addition, all prediction buffers were set to 
zero for this baseline study. The cause of the third borderline LOS (4.989) could not be precisely determined but 
may have also involved a turn modeling issue. In all three cases, the use of prediction buffers and/or tactical CR 
capability would have prevented the borderline penetrations. The distribution of actual CPAs describes the 
effectiveness of the ASAS system and the constraints posed by the traffic density. 

The CPA distributions show that many aircraft end up being separated by much more than the minimum 
requirement of 5 NM. This result reflects the notion of a busy airspace, in which a given aircraft that resolves a 
conflict may subsequently encounter multiple additional conflicts at later times. The end result can be multiple 
resolution maneuvers that further 
separate the initial conflicting pair. 

As traffic density increases, a 
greater spread in the CPA 
distribution is observed. For 
example, at the lowest traffic 
density condition, few CPAs were 
recorded beyond 1 1 NM. At higher 
traffic density conditions, a number 
of large separation distances 
(beyond 11 NM) are recorded, as 
well as an increase close to the 
minimum value of 5 NM. 

Distribution of distances shows a 
marked “peak” between 5 and 
7 NM suggesting that as the 
airspace gets saturated, aircraft are 
constrained to fly closer to the 
separation minima. At lower 
densities there seems to be a more 
uniform spread of values. 

The median conflict alert duration is shown in Figure . It represents the time betwee conflict detection and 
the time the conflict alert cleared, i.e. the conflict was resolved. This metric reflects the effectiveness of the 
resolution automation and the constraints imposed by traffic density in finding suitable resolutions. The increasing 
durations follow the increasing traffic density. 

Conflicts were classified as first generation and second generation, where a second generation conflict is one that 
is created as a result of 
solving a previous conflict. 

This is referred to as 
“secondary conflicts” in the 
studies referenced earlier 5,6 
Second generation conflicts 
are further classified based on 
the conditions preceding the 
conflicts. In particular, the 
so-called “sidewalk” and 
“coincidence” conflicts were 
investigated in this 
experiment. Repeated 

conflicts between the same 
pair of aircraft, caused by 
simultaneous resolutions of Figure 7. Median Conflict Alert Duration 

previous conflicts are 
identified as sidewalk 

conflicts. Coincidence conflicts are the result of two aircraft simultaneously solving unrelated conflicts. These 
conditions, which are the result of distributed decision making, seem to have a very low probability, even in the 
rigorous laboratory conditions of this experiment. Out of nearly 2800 simulated flight hours and nearly 6000 
conflicts in this study, only 11 conflicts were identified as possible coincidence conflicts and no sidewalk conflcits 
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were observed. With initial detection occurring near 10 minutes from predicted LOS, all of these cases were safely 
resolved. 


VI. Future Work 

As mentioned earlier, this experiment is the first of a series of studies designed to analyze and evaluate the safety 
performance of airborne separation. The next set of studies will seek to evaluate safety performance in similar 
simulation conditions introducing one element of uncertainty at a time to understand its individual effect. A set of 
exploratory runs were performed to begin the design of this next set of experiments. The initial observations were 
compared with the baseline run described in this paper in which no prediction errors or uncertainties were modeled. 
This means ADS-B reception was perfect, full intent was shared, the pilot model responded at “normal” rates, and 
wind predictions were accurate. The exploratory runs included the following configuration settings and were based 
in all cases on a mean traffic density of 8.61 aircraft per 10000 NM2 (about 170 flights over a four hour run) and an 
average of 46 flight hours. The following conditions were individually tested. 

1. Pilot Model: The pilot model was configured to have a sensorial delay so that the reaction time to a conflict alert 

was longer than the “normal” case. The “sleepy” sensor delay mean and standard deviation used were 2000 and 
2.0 milliseconds respectively. 

2. ADS B signal interference: The ADS-B model was configured to have default signal degradation settings due to 

interference and distance as described in 18,19 

3. ADS-B intent exchange: Two runs, one sending four trajectory change points (TCP) and one sending two TCPs 
for intent transmission were performed. Typical CR maneuvers require three to four TCPs to define the 
complete maneuver.. 

4. Wind prediction error: Four runs were performed to test a modest wind speed prediction error of 10 NM/hour. 
For the first two runs, no attempt was made to compensate for the resulting trajectory prediction uncertainty. 
For the second set of runs, a longitudinal buffer was used to increase the likelihood of conflict detection. 

A set of metrics were collected to begin to probe the system’s safety performance in the presence of these 
limitations and uncertainties. The graph in Figure 3 shows the observed effect on conflict counts relative to the 
baseline. No statistics were possible given the limited amount of data collected. The percentage of change in the 
observed number of conflicts in the PM and 4-TCP runs seems to indicate little effect. The ADS-B interference and 
2-TCP runs exhibit an observed 
increase in conflict detections of 
about 20%. In all four runs there 
were no losses of separation. 

The first set of wind 
prediction error runs showed a 
slight increase in the number of 
observed conflicts detections. In 
addition, there were three losses 
of separation, with CPAs of 4.94, 

4.98 and 4.90 NM. During the 
second set wind prediction error 
runs that employed longitudinal 
buffers, the observed number of 
conflict detections were higher, 
but there were no losses of 
separation. This is just a small Figure 3. Exploratory Runs 

sample of on-going research; 

many more exploratory runs will be conducted to guide the next set of experiments in the SPAS study. 

VII. Conclusion 

The SPAS experiment suite is a methodical, rigorous, and incremental approach to the study of the safety 
implications of prediction errors and system uncertainties on airborne separation. This study is unique in that it 
provides new research results based on medium fidelity simulation systems, with key sub-systems such as the ASAS 
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Percentage of Change with Respect to Baseline Conflict Count 




and ADS-B modeled in high fidelity. Early results at five times the typical traffic density in today’s NAS indicate 
that a distributed approach to maintaining separation of aircraft can be safely achieved, even after introducing 
several sources of error and uncertainty. Research scheduled within the next two years, examining higher levels of 
density, multiple altitudes, other sources of error and uncertainty, and coordinating with a centralized approach, will 
add greatly to the knowledge of methodologies to safely automate the separation of aircraft. 

Acknowledgments 

The research team acknowledges the indispensable work by our ATOS simulation development team Daniel 
Finkelsztein, John Bunell, Doug Mielke, Robert Vivona, David Karr, David Roscoe, and Frank Bussink, and valued 
assistance from Ed Scearce, Jeff Viken, Jeremy Smith, Dave McNally, and Mike Paglione. 


References 

1 Joint Planning and Development Office: Concept of Operations for the Next Generation Air Transportation System.”, 
Version 2.0, June 2007. http://www.ipdo.gov/librarv/NcxtGcn v2.0.pdf 

2 FAA/Eurocontrol Cooperative R&D Action Plan 1, “Principles of Operation for the Use of Airborne Separation Assurance 
Systems.”, Version 7.1, June 2001. http://www.eurocontrol.int/care-asas/gallerv/content/public/docs/po-asas71.pdf 

3 FAA System Safety Handbook, Chapter 9: Analysis Techniques, December 30, 2000 

4 Stephens, Richard, A. and Talso, Warner, System safety Analysis Handbook: A Source Book for Safety Practitioners, 
System, Safety Society, 2nd Edition, August 1999. 

5 Bilimoria,K., Sheth, H., Grabble, S., “Performance Evaluation of Airborne Separation Assurance for Free Flight,” AIAA 
Guidance, Navigation, and Control Conference and Exhibit, AIAA 2000-4269, August 2000 

6 Krozel, J., Peters, M., Bilimoria, K., Lee, C., and Mitchell, J., “Systems Performance Characteristics of Centralized and 
Decentralized Air traffic Separation Strategies,” 4 th . USA/Europe Air Traffic Management R7D Seminar, Santa Fe, NM, 
December 200 1 . 

7 Andrews, J., Erzberger, H., and Welch, J., “Safety Analysis for Advanced Separation Concepts,” Air Traffic Control. 
Quarterly, Vol. 14, No. 1, 2006, pp. 5-24 

8 Finkelstein, D„ Lung, Vivona, R„ Bunell, J., Mielke D„ and Chung W.: “Airspace and Traffic Operations Simulation for 
Distributed ATM Research and Development.” AIAA 2005-6488. 

9 Bussink, Hoekstra, and Heesbeen: “Traffic Manager: A Flexible Desktop Simulation Tool Enabling Future ATM Research.” 
DASC 2005, pp. 3.B.4-1 to 3.B.4-10. 

10 Kan', D., Roscoe, D., and Vivona R., “An Integrated Flight-Deck Decision-Support Tool in an Autonomous Flight 
Simulation,” AIAA Modeling and Simulation Technologies Conference and Exhibit, AIAA-2004-5261, August 2004 

11 Kan, D., Roscoe, D., and Vivona R., “Conflict Detection Using Variable 4D Uncertainty Bounds to Control Missed 
Alerts,” AIAA Guidance, Navigation, and Control Conference and Exhibit, AIAA-2006-6057, August 2006 

12 Wing, D., Bannore, B., and Krishnamurthy, K., “Use of Traffic Intent Infonnation by Autonomous Aircraft in Constrained 
Operations,” ALIA Guidance, Navigation, and Control Conference and Exhibit, AIAA-2002-4555, August 2002. 

13 RTCA Special Committee 186: “Application of Airborne Conflict Management: Detection, Prevention, & Resolution.” 
RTC A/DO-263, December 2000. 

14 Wing, D.J., Krishnamurthy, K., Barhydt, R., and Bannore, B., “Pilot Interactions in an Over-constrained Conflict Scenario 
as Studied in a Piloted Simulation of Autonomous Aircraft Operations,” 5th USA/Europe Air Traffic Management R&D 
Seminar (ATM2003), Budapest, Hungary, June 2003. 

15 Vivona, R., Karr, D., and Roscoe, “Pattern-Based Genetic Algorithm for Airborne Conflict Resolution,” AIAA Guidance, 
Navigation, and Control Conference and Exhibit, AIAA-2006-6060, August 2006 

16 Jesse Aronsonf, Vikram Manikonda*, Wilbur Peng*, Renato Levy* and Karlin RothJ, “An HLA Compliant Agent-Based 
Fast-Time Simulation Architecture For Analysis Of Civil Aviation Concepts,” Spring SISO Simulation Interoperability 
Workshop, April, 2003 

17 Paglione, M., Oaks, R. and Bilimoria, K., “Methodology for Generating Conflcit Scenarios by Time Shifting Recorded 
Traffic Data f AIAA 3 rd . Aviation Technology , Integration and Operations Conference, AIAA 2003-6724, Denver, Colorado, 
November 20003 

18 Chung, W. and Staab, R. “A 1090 Extended Squitter Automatic Dependent Surveillance - Broadcast (ADS-B) Reception 
Model for Air-Traffic -Management Simulations,” AIAA 2006-6614, Modeling and Simulation Technologies Conference and 
Exhibit, August 2006, 

19 Barhydt, R., Palmer, M., Chung, W., and Loveness, G., “ADS-B within a multi-aircraft simulation for distributed air- 
ground traffic management,” Digital Avionics Systems Conference (DASC), 2004. 


10 

American Institute of Aeronautics and Astronautics 



